1. Field of the Invention
The present invention relates to an image pickup apparatus that obtains a two-dimensional light intensity distribution and angle information of a ray in an object space, and an image processing method for the image obtained by the image pickup apparatus.
2. Description of the Related Art
Recently, an image pickup apparatus that performs a calculation using data obtained by an image pickup element and performs a corresponding digital image processing so as to output various images is proposed. Ren Ng, et al., “Light Field Photography with a Hand-held Plenoptic Camera”, 2005 Computer Science Technical Report CTSR, Todor Georgiev, et al., “Superresolution with Plenoptic 2.0 Camera”, 2009 Optical Society of America, and Aaron Isaksen, et al., “Dynamically Reparameterized Light Fields” and ACM SIGGRAPH and pp. 297-306 (2000), disclose an image pickup apparatus that obtains the two-dimensional light intensity distribution and the angle information of the ray in the object space using “Light Field Photography”. The two-dimensional light intensity distribution and the angle information of the ray are collectively referred to as a light field, and three-dimensional information in the object space can be obtained by obtaining the light field. According to such an image pickup apparatus, the light field is obtained and an image processing is performed after taking an image so that a focus position of the image called a refocus, a shot point of view, a depth of field, or the like can be changed.
However, such an image pickup apparatus needs to use pixels of the image pickup element for storing the angle information of the ray, in addition to storing the two-dimensional light intensity distribution. Therefore, the spatial resolution is deteriorated with respect to an image pickup apparatus which only stores the two-dimensional light intensity distribution. Todor Georgiev, et al., “Superresolution with Plenoptic 2.0 Camera”, 2009 Optical Society of America, discloses a configuration in which a certain point on an image plane that is formed by an imaging optical system is taken by a plurality of small lenses constituting a lens array. A plurality of small images obtained like this are reconstructed so that the resolution of the reconstructed image can be improved. Such a method of improving the resolution is referred to as a “pixel shift effect”.
However, Todor Georgiev, et al., “Superresolution with Plenoptic 2.0 Camera”, 2009 Optical Society of America, only describes a specific focus position as a method of obtaining the effect of the super-resolution from subpixel shift. When an image is generated at a different focus position in accordance with the refocus, the effect of the super-resolution from subpixel shift is changed in accordance with the focus position and the spatial resolution is deteriorated.